Multi-lingual closed-captioning

ABSTRACT

A terminal for a television broadcast system with multi-lingual closed captioning support comprises a television receiver and a decoder for decoding a closed-captioning information stream. The terminal is programmed to continuously form data elements from the embedded closed-captioning information stream, each data element being unique to a closed-caption string. The terminal is connected to a network ( 29 ) comprising a translation service server ( 30 ), and is programmed to send the data elements with a specification of a desired caption language to the translation service server, and to receive captions in the desired language.

FIELD OF THE INVENTION

The invention relates to a terminal for a television broadcast systemwith multi-lingual closed captioning support, comprising a televisionreceiver, a decoder for decoding an embedded closed-captioninginformation stream, which terminal is programmed to continuously formdata elements from the embedded closed-captioning information stream,each data element being unique to a closed-caption string.

The invention further relates to a television broadcast system withmulti-lingual closed-captioning support.

BACKGROUND OF THE INVENTION

An example of a terminal as described in the opening paragraph is known.The abstract of TW-A-303 568 discloses a method that involves receivinga television signal containing caption data and fetching caption datafrom the television signal to form a referenced image signal and acaption data signal. An input instruction is received with respect toselected caption data and the selected caption data is stored. Textmeaning and/or translation data corresponding to the selected captiondata stored in memory is fetched, displayed on the screen, and theselected caption data is translated and explained.

The known terminal has a number of drawbacks. Because a translation ofthe stored captions takes place on the terminal, the terminal must beprovided with the appropriate dictionary and translation software. Tomake the terminal suitable for several languages requires multipledictionaries to be present. These must be stored. Additionally, becausethe data elements formed from the closed-captioning stream, i.e. thecaption data signal itself, is stored first, captions for a program onlybecome available in the desired language after the program has beenbroadcast.

OBJECT AND SUMMARY OF THE INVENTION

It is an object of the invention to provide an improved terminal of thetype mentioned in the pre-amble of claim 1, which allows multi-lingualclosed captioning support in any of a large number of possiblelanguages.

This object is achieved by the terminal according to the invention,which is characterised in that the terminal further comprises abidirectional network interface for connection to a network comprising atranslation service server, and is programmed to send the data elementswith a specification of a desired caption language to the translationservice server, and to receive captions in the desired language from thetranslation service server.

Using such an architecture, the translated captions can be providedexternally, namely by the translation service server. Thus, any numberof translation service servers can be used, each providing support for adifferent language, or one powerful server can be used that providessupport for a large number of languages. Extensive changes or downloadsare not required when support for a further language is to be provided,or when account needs to be taken of evolutions in an existing language.

Preferably, the terminal is further programmed to establish anend-to-end connection to the translation service server through which itre-directs the formed data elements and receives the captions in thedesired language.

Thus, continuous support can be provided whilst a program is beingbroadcast. It is not necessary to store an entire program receivedthrough the broadcast channel until such time as the captions in thedesired language are made available to the terminal.

In a preferred embodiment of the invention, the terminal is programmedto include at least one return address, specifying a caption receivingterminal with the formed data elements.

Thus, the captions in the desired language can be made available on adifferent terminal as well as or instead of the terminal sending thedata elements. It becomes possible for viewers who do not have alanguage in common with which they are both familiar, to view a programtogether. One of them will receive the captions in a language desired byhim on the caption-receiving terminal, this could be a computer ormobile phone with an Internet browser, for example. The other viewer canreceive the captions in his language on the terminal for the televisionbroadcast system.

According to a further aspect of the invention, a television broadcastsystem with multi-lingual closed-captioning support is provided,comprising at least one terminal according to any one of claims 1-8,which system further comprises a network to which the terminal isconnected through its interface, the network comprising a translationservice server on which translation software is installed fortranslating text strings received from the terminal into a desiredlangauage specified by the terminal, wherein the terminal is programmedto continuously form text strings from the embedded closed-captioninginformation stream and to re-direct the formed text strings through thenetwork to the translation service server.

The system is particularly suited to using conventional translationsoftware, through its use of strings. In particular, an Internet-basedtranslation service can be part of the system. Such a service compriseslarge dictionaries and is configured for handling a large number ofrequests, such as would be made in a system comprising many terminals.In addition, such a translation service is usually free of charge.

According to another aspect of the invention, a television broadcastsystem with multi-lingual closed-captioning support is provided,comprising at least one terminal according to any one of claims 1-8,which system further comprises a network to which the terminal isconnected through its interface, the network comprising a translationservice server with a caption database with entries for a plurality ofcaptions, each entry recording a caption identifier and at least onetranslation of a caption string, wherein the terminal is programmed tore-direct caption identifiers formed from the decoded closed-captioninginformation through the network to the translation service server.

Such a system can be advantageously used by broadcasters to providemulti-lingual closed-captioning support for a much wider range oflanguages than would be possible if the captions were all to be providedwith the closed-captioning information stream embedded in the broadcastsignal. The wide range of possible languages even allows the provisionof closed captions in a plurality of regional dialects. Addition of anextra language or dialect merely requires an addition to the captiondatabase.

BRIEF DECRIPTION OF THE DRAWINGS

The invention will now be explained in further detail with reference tothe accompanying drawings, of which

FIG. 1 is a schematic diagram for explaining embodiments of thebroadcast systems according to the invention;

FIG. 2 shows a diagram of some key hardware components of a terminalaccording to the invention; and

FIG. 3 shows a basic architecture for embodiments of the terminalaccording to the invention.

DESCRIPTION OF PREFERRED EMBODIMENTS

Two examples of terminals for use in connection with the invention areshown in FIG. 1. The first example is a set-top box 1, the secondexample an interactive digital television set IDTV 2). The terminalcomprises a television receiver, for receiving broadcast informationfrom a broadcaster 3 through a broadcast network 4. The broadcastnetwork 4 can be a cable network, satellite network or terrestrialbroadcast network. Broadcast information can be digital or analogue(e.g. PAL, NTSC, SECAM, DVB).

Turning to FIG. 2, the terminal comprises a broadcast channel connection5 for connection to the broadcast network 4. The terminal comprises atuner 6 for tuning in to a specific carrier frequency. In the example ofFIG. 2, the terminal is capable of receiving analogue and/or digitalbroadcast streams. In the former case, the signal is passed from thetuner 6 to an analogue video processor 7. In the latter case, the signalis first de-modulated and de-multiplexed in a demodulator 8 anddemultiplexer 9, respectively. Digital broadcasts are usuallycompressed, for example using the MPEG2-standard compression algorithm,so the terminal comprises an MPEG video decoder 10 for retrieving thebroadcast data. The terminal further comprises a system processing unit11 for processing broadcast and video data. System memory 12, connectedto the processing unit 11 via a memory controller 13 and a bus, can beused to temporarily store the video and broadcast data. A display engine14 outputs video data in a format suitable for displaying on a displayconnected through a video output channel 11. In the set-top box 1 therewould be a digital video interface and/or an analogue interface to aconventional television 16 or a video recorder (not shown in FIG. 1)connected to the set-top box 1. Similarly, an audio engine 17 providesan audio signal through an audio output channel 18.

The processing unit 11 is also capable of decoding closed-captioninginformation streams that are embedded in the broadcast data receivedthrough the broadcast channel connection 5. Captions are text locatedsomewhere in a video picture. Closed captions are captions that arehidden in the video signal, invisible without a decoder.

The exact way in which the closed-captioning information is embedded inthe video signal received through the broadcast channel connection 5depends on the broadcast standard. In analogue television broadcasts,closed captions are hidden in teletext pages, usually with page number888. The teletext pages are broadcast in the virtual blanking intervalin case of interlaced broadcasts. In digital television according to,for example, the Digital Video Broadcasting Standard, closed captioninginformation is broadcast as MPEG2-packets with the closed captioninginformation specifically identified as such.

The terminal according to the invention further comprises abidirectional network interface. This means that it is capable of beingconnected to a network, through which it can both transmit and receivedata. In other words, there is a second channel, a return channel,physically separate from the broadcast channel 5. Thus, the terminalcould, for example, be compliant with the IB1 profile laid down in theMultimedia Home Platform specification for digital video broadcasting.

As an example of an implementation of the bidirectional networkinterface, the terminal illustrated in FIG. 2 comprises a PCI(Peripheral Component Interface)-controller 19, and an ethernet-card 20,connected to a PCI-bus. The terminal further comprises a USB (UniversalSerial Bus)-controller 21. The skilled person will realise that abi-directional network interface could also, for example, be realised byconnecting an external modem or network card to a USB port or throughthe use of a PCI-modem.

The terminal of FIG. 2 further comprises an I2C-controller 22 and anEIDE-controller 23. A hard disk could be attached to the latter. Theuser can issue commands to the terminal through a remote controlcommunications channel 24, using a remote control unit (not shown), forwhich purpose an IR-controller 25 is provided. It will be realised thatthe terminal will comprise further components, which are not relevant inthe present context. The various components can be integrated to ahigher or lesser degree in one or more integrated circuits in theterminal, or indeed be present as software modules, which can be run onthe system processing unit 11, so as to provide the equivalentfunctionality.

For the purpose of explaining the invention, a viewer's home isindicated by a dashed line 26. The viewer has a home network at hisdisposal. Both the set-top box 1 and the IDTV 2 are connected to thehome network, for example through the ethernet-card 20. Additionally,the home network comprises a server 27 and an Internet access router 28.The Internet access router 28 provides access to the Internet 29, whichof course comprises multiple servers, an example being indicated byreference number 30.

Conventional closed-captioning services provide only limitedmulti-lingual support. The broadcaster 3 might occasionally provideclosed-captioning information in a couple of languages with thebroadcast signal. However, the number of languages is usually limited toa few important languages. To provide closed captions for a large numberof languages as an embedded information stream would require a lot ofbandwidth. Each terminal would receive all the different languageversions, even though only one is required. Also, there is no incentivefor the broadcaster 3 to provide closed-captioning support for a largenumber of languages. It would be much more efficient if the viewer wereto be able to retrieve the captions in the desired language himself. Theinvention provides a terminal and system by means of which this can beachieved.

The terminal is programmed to continuously form data elements from theclosed-captioning information stream received through the broadcastchannel 5. As will be explained below, these data elements can comprisestrings encoding caption text or identifiers identifying a caption text,or a combination. Because the terminal comprises a bi-directionalnetwork interface, it is able to send these data elements to atranslation service server. The translation service server returns thecaptions in the desired language, which the terminal receives throughits network interface.

The use of a translation service server has the advantage that thirdparties can provide captions in various languages, possibly charging forthe service. The broadcaster 3 can make the captions available in alarge number of languages, without using up the bandwidth allocated tohim in the broadcast network 4.

According to one embodiment of the invention, the terminal is programmedto continuously form text strings from the embedded closed-captioninginformation stream and to re-direct the formed text strings to atranslation service server. Translation software is installed on thetranslation service server for translating the text strings receivedfrom the terminal into the desired language specified by the terminal.

Both the server 27 in the home network and the server 30 in the Internet29 can be used as translation service servers. The advantage of usingthe server 27 in the home network is that communication costs arereduced. It is an implementation of the invention that is well suited tothose needing daily translation of broadcast material. The advantages ofusing a server 30 in the Internet 29, is that there are already a numberof Internet-based translation services providing free translation. Inaddition, the number of languages supported can be easily increased. Itis not necessary for the viewer to have translation software thatsupports many languages.

The desired language can be specified by the destination address towhich the text strings are sent, or by adding a code to the text stringsthat the translation software is able to recognise. In the former case,a uniform resource locator (URL) could be used for each language.

It would be possible to provide a URL for each combination of sourcelanguage—the language in which the text strings are provided to thetranslation service server—and desired language. The terminal would thenneed to have access to a list of addresses. Alternatively, especially inthe home network, a standard language could be assumed, so thatinformation regarding the source language need not be sent to thetranslation service server at all. However, to be able to provide atruly universal translation service from any language to any language,the terminal is preferably programmed to add a specification of a sourcelanguage to the formed text strings re-directed to the translationservice server. The embedded closed-captioning information streamprovided through the broadcast network 4 can thus be in any language. Itis conceivable that the specification of the source or destinationlanguage is provided implicitly, in that the server 30 is able todetermine the geographical location of the terminal from a returnaddress or from a network address revealed to it when an end-to-endconnection is set up between it and the terminal. It is also conceivablethat the source languages could be derived from an Electronic ProgramGuide (EPG) web service or similar service.

If the Internet-based server 30 is used as translation service server,an alternative embodiment of the invention is possible, which reducesthe amount of data traffic. In this embodiment, the server 30 comprisesa caption database with entries for a plurality of captions, each entryrecording a caption identifier and at least one translation of a captionstring. The terminal is programmed to re-direct caption identifiersformed from the decoded closed-captioning information through theInternet 29 to the server 30. Specification of the desired language bythe terminal can be carried out in the same ways as in the embodiment inwhich text strings are re-directed to the translation service server.

The broadcaster 3 translates the captions into a number of languages andcompiles the translated captions in a database hosted on the server 30.Because only caption identifiers are sent to the server 30, the amountof network traffic between the terminal and the server 30 is reduced. Itis more advantageous for the broadcaster 3 to provide the translatedcaptions in this way than as an embedded information stream through thebroadcast network 4. The broadcaster 3 can use the, often limited,bandwidth of the broadcast network 4 for other purposes. He can chargeseparately for the service of providing translated captions, evendifferentiating between languages.

It would be conceivable to send the data elements formed by the terminalfrom the embedded closed-captioning stream as an electronic mail messageto the server 30 in the Internet 29. However, the transmission protocolsusually used for sending electronic mail make use of gateways that slowdown the messages transferred from the terminal to the translationservice server. Additionally, the terminal would have to use a mailfetching protocol to retrieve the returned captions in the desiredlanguage. It would be quite difficult for the terminal to synchronisethe captions in the desired language with the video signal receivedthrough the broadcast channel 5. The terminal would have to buffer alarge amount of video data until the captions in the desired languagebecome available.

The terminal is therefore preferably programmed to establish anend-to-end connection to the translation service server through which itre-directs the formed data elements and receives the captions in thedesired language. The HyperText Transfer Protocol running over theTransmission Control Protocol can, for example, be used to this end.

It is not necessary that the captions in the desired language bereturned to the terminal that formed the data elements from the embeddedclosed-captioning information stream. In one embodiment of theinvention, the terminal is programmed to include at least one returnaddress specifying a caption-receiving terminal with the formed dataelements. The caption-receiving terminal need not be a terminal such asthe IDTV 2 or the set-top box 1. It need only be capable of receivingthe captions in the desired language and making them available to theuser. Generally, it need not be capable of receiving the broadcast datafrom the broadcaster 3. For example, FIG. 1 shows a mobile phone 31 witha web-browser, which can receive the captions in the desired languageand display them. If, for example, two people are watching the IDTV 2together, one can receive captions in one language on the mobile phone31, whilst the other makes use of captions in a different languagedisplayed on the IDTV 2. Another advantage of using a separatecaption-receiving terminal is that it is possible to view the entirepicture on the IDTV 2, without the captions getting in the way.

According to yet a further embodiment of the invention, the terminalcomprises a speech synthesiser, capable of converting received captionsin the desired language into an audio signal. This embodiment can beimplemented in various ways. For example, where the translation serviceserver comprises a caption database, some or all of the captions can bein an audio format. Alternatively, the terminal can have been providedwith software to synthesise an audio signal from strings provided by thetranslation service server, which audio signal is made available throughthe audio output channel 18.

At least one embodiment of the terminal according to the inventioncomprises a display reformattor, capable of adding the received captionsin the desired language to a television picture received by thetelevision broadcast receiver. Although the foregoing description mayhave suggested that the captions in the desired language consist ofcharacter strings, this need not be the case. Indeed, character stringsby themselves would not provide a very pleasant viewing experience,since they would be displayed in a default location in the picture witha default appearance.

Therefore, the terminal is preferably programmed to use meta-dataprovided with the received captions in the desired language to determinethe appearance of the received captions. The meta-data can encode suchparameters as the colour of the caption, its location in the picture,the length of time it is to be displayed, etc. It is conceivable thatthe captions are provided as strings with tagged on codes containing themeta-data. Alternatively, the captions in the desired language can besent in the shape of picture files, encoding such things as colour. Anexample of a file-format that can be used in this connection is theGraphical Interchange Format (GIF).

Where use is made of a translation service server with a captiondatabase, the meta-data can be included in the database, especially ifthe broadcaster 3 is responsible for the contents of the database. Whereuse is being made of a translation service server that comprisestranslation software, this would be more difficult. It would thereforebe advantageous for the terminals to be programmed to add meta-dataspecifying the appearance of a caption to the formed data elementsredirected to the translation service server. The terminal can use theclosed-captioning information stream provided through the broadcastchannel 5 for this purpose. Meta-data specifying the appearance ofcaptions can be included with the closed-captioning text in nearly allthe formats defined for broadcasting closed-captioning information. Theterminal need only be programmed to extract this meta-data and add it tothe text strings or caption identifiers that it sends to the translationservice server. The terminal could also edit the meta-data beforesending it to the translation service server. For example, it could usea colour mapping to alter the colours specified by the broadcastmeta-data It could thus take account of the colour-blindness of aviewer, or of the background colour in the picture, for example.

As a further feature of the invention, the specification of a desiredlanguage sent by the terminal to the translation service server mayinclude an identification of a subset of the desired caption language.For example, this feature could be used in combination with a parentalguard function of the IDTV 2 or set-top box 1, to limit the exposure ofchildren to rude or violent language. Where use is made of a captiondatabase on the server 30, the database can comprise an adult and anunder-age set of captions. Where use is made of translation software onone of the servers 27, 30, the software could use different subsets of adictionary. This feature could also be used to take account of regionallanguage differences and local dialects. The broadcaster 3 can provide acaption database on the server 30 comprising captions in many differentregional variants of a language. In the alternative embodiment of theinvention, the translation software can make use of subsets of adictionary.

It will be realised that the multi-lingual closed captioning supportprovided by the invention can also be used in connection with recordedbroadcast data. In one further developed embodiment of the invention,the terminal comprises, or is capable of being connected to, a playerfor a portable multi-media data storage means with embeddedclosed-captioning information stored thereon. This embodiment of theterminal is programmed to continuously form data elements from theembedded closed-captioning information stream stored on the multi-mediadata storage means, each data element being unique to aclosed-captioning string.

The portable multi-media data storage means could be a Digital VersatileDisk (DVD), a Compact Disk (CD), a hard disk, or the like. The playercan be an internal player, accessible through the EIDE-controller 23 orthe I2C-controller 22, for instance. Alternatively, an external playercan be attachable to the terminal, for example to a USB-port, with databeing transferred through use of the USB-controller 21.

This embodiment of the invention is useful for use with CDs, whichgenerally do not have the capacity for storing video data with a largenumber of captions. It is also useful for use with DVDs, which usuallydo comprise caption information, but only in a limited number oflanguages.

All embodiments of the terminal according to the invention can beadvantageously provided by means of software applications, executed bythe system processing unit 11. FIG. 3 shows the architecture of onewidespread type of terminal, conformant to the Multimedia Home Platform(MHP) standard. At the lowest layer, the terminal comprises resources32, which may differ from terminal to terminal, the configuration ofFIG. 2 being merely an example. On top of that there is a layer ofsystem software 32 and an application manager 33, which function as anoperating system, providing a standard, well-defined ApplicationProgrammin Interface (API) to applications 34, which form the top layer.The application manager 33 can, for example, be a Java Virtual Machine,for interpreting so-called Xlets. The Xlets are applications 34 writtenin Java code. The invention may be implemented as one or more Xletsthat, when run on a terminal with the architecture of FIG. 3, providethe terminal with the functionality described above.

It will be apparent to those skilled in the art that the invention isnot limited to the above-described embodiments, which can be variedwithin the scope of the attached claims. For instance, the translationservice server and terminal can exchange data elements and captions in aproprietary format or make use of a universal standard for dataexchange.

Disclosed is a terminal for a television broadcast system withmulti-lingual closed captioning support comprises a television receiverand a decoder for decoding a closed-captioning information stream. Theterminal is programmed to continuously form data elements from theembedded closed-captioning information stream, each data element beingunique to a closed-caption string. The terminal is connected to anetwork (29) comprising a translation service server (30), and isprogrammed to send the data elements with a specification of a desiredcaption language to the translation service server, and to receivecaptions in the desired language.

1. Terminal for a television broadcast system with multi-lingual closed captioning support, comprising a television receiver, a decoder for decoding an embedded closed-captioning information stream, which terminal is programmed to continuously form data elements from the embedded closed-captioning information stream, each data element being unique to a closed-caption string, characterised in that the terminal further comprises a bi-directional network interface (19, 20) for connection to a network (29) comprising a translation service server (27, 30), and is programmed to send the data elements with a specification of a desired caption language to the translation service server (27, 30), and to receive captions in the desired language from the translation service server (27, 30).
 2. Terminal according to claim 1, wherein the terminal is further programmed to establish an end-to-end connection to the translation service server (27, 30) through which it re-directs the formed data elements and receives the captions in the desired language.
 3. Terminal according to claim 2, wherein the terminal is programmed to include at least one return address, specifying a caption receiving terminal (1, 2, 31) with the formed data elements.
 4. Terminal according to claim 1, wherein the specification of a desired language includes a specification of a subset of the desired caption language.
 5. Terminal according to claim 1, comprising a speech synthesiser, capable of converting received captions in the desired language into an audio signal.
 6. Terminal according to claim 1, comprising a display reformattor capable of adding the received captions in the desired language to a television picture received by the television receiver, wherein the terminal is programmed to use meta-data provided with the received captions in the desired language to determine the appearance of the received captions.
 7. Terminal according to claim 1, programmed to add meta-data, specifying the appearance of a caption, to the formed data elements redirected to the translation service server (27, 30).
 8. Terminal according to claim 1, comprising or capable of being connected to, a player for a portable multi-media data storage means with embedded closed-captioning information stored thereon, and programmed to continuously form data elements from the embedded closed-captioning information stream, each data element being unique to a closed-caption string.
 9. Television broadcast system with multi-lingual closed-captioning support, comprising at least one terminal (1, 2) according to claim 1, which system further comprises a network (29) to which the terminal (1, 2) is connected through its interface (19, 20), the network (29) comprising a translation service server (27, 30) on which translation software is installed for translating text strings received from the terminal (1, 2) into a desired langauage specified by the terminal (1, 2), wherein the terminal (1, 2) is programmed to continuously form text strings from the embedded closed-captioning information stream and to re-direct the formed text strings through the network (29) to the translation service server (27, 30).
 10. Television broadcast system according to claim 9, wherein the terminal (1, 2) is programmed to add a specification of a source language to the formed text strings re-directed to the translation service server (27, 30).
 11. Television broadcast system with multi-lingual closed-captioning support, comprising at least one terminal (1, 2) according to claim 1, which system further comprises a network (29) to which the terminal (1, 2) is connected through its interface (19, 20), the network (29) comprising a translation service server (30) with a caption database with entries for a plurality of captions, each entry recording a caption identifier and at least one translation of a caption string, wherein the terminal (1, 2) is programmed to re-direct caption identifiers formed from the decoded closed-captioning information through the network (29) to the translation service server (30).
 12. Computer program for a terminal (1, 2) comprising a television receiver, a decoder for decoding an embedded closed-captioning information stream, a bidirectional network interface (19, 20) for connection to a network (29) and a central processing unit (11), which computer program, when run on the terminal (1, 2), is capable of providing the terminal (1, 2) with the functionality of a terminal (1, 2) in a television broadcast system according to claim
 9. 13. Data storage means comprising a computer program according to claim
 12. 